Emotion recognizing method, sensibility creating method, device, and software

ABSTRACT

An object of the invention is to provide an emotion detecting method capable of detecting emotion of a human accurately, and provide sensibility generating method capable of outputting sensibility akin to that of a human. An intensity, a tempo, and intonation in each word of a voice are detected based on an inputted voice signal, amounts of change are obtained for the detected contents, respectively, and signals expressing each states of emotion of anger, sadness, and pleasure are generated based on the amounts of change. A partner&#39;s emotion or situation information is inputted, and thus instinctive motivation information is generated. Moreover, emotion information including basic emotion parameters of pleasure, anger, and sadness is generated, which is controlled based on the individuality information.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to an emotion detecting method, asensibility generating method, a system of the same and software forexecuting the same. The emotion detecting method of the presentinvention can be utilized for emotion detection in a medical field andfor a variety of systems as a part of artificial intelligence andartificial sensibility. Furthermore, a sensibility generating method ofthe present invention can be utilized for a variety of systems used inmany ways for controlling the sensibility of virtual humans and robots.

[0003] 2. Description of the Related Art

[0004] Conventional arts related to the emotion detecting method of thepresent invention have been disclosed in, for example, JapaneseUnexamined Patent Application Publication Nos. Hei5-12023, Hei9-22296,and Hei11-119791.

[0005] Japanese Unexamined Patent Application Publication No. Hei5-12023discloses that the continuation time of voice, the formant frequency ofvoice, and the intensity of voice for each frequency are respectivelydetected as amounts of characteristic of the voice. Furthermore, thisgazette also discloses that a difference between a reference signal andthe respective amounts of characteristic is detected and emotiondetection is made by fuzzy inference based on the detected differenceamount.

[0006] Japanese Unexamined Patent Application Publication No. Hei9-22296discloses that a generating rate of voices (the number of mora per unittime), a voice pitch frequency, sound volume, and voice spectrum aredetected as amounts of characteristic of the voice. Furthermore, thisgazette also discloses that emotions are detected based on the detectedamounts of characteristic and results obtained by statisticallyprocessing HMM (Hidden Markov Model).

[0007] Japanese Unexamined Patent Application Publication No.Hei11-119791 discloses that emotions are detected based on a probabilityof phoneme spectrum in its transition state by utilizing HMM.

[0008] On the other hand, as conventional arts related to thesensibility generating method of the present invention, for example,“Emotion Generating System and Emotion Generating Method” disclosed inJapanese Unexamined Patent Application Publication No. Hei11-265239 isknown.

[0009] Emotions which express the internal states of humans and the likechange variously depending on situations at that time. JapaneseUnexamined Patent Application Publication No. Hei11-265239 discloses thetechnology for realizing generation of emotions in unpredictablesituations.

[0010] Specifically, situations are evaluated in view of the predictablesituations, and system's own emotion is generated. In addition, emotionsthat were actually generated in the past and situations at that time areanalyzed, and unpredictable collateral conditions peculiar to therespective situations and emotions corresponding thereto are learned.When a situation newly inputted satisfies the collateral conditions,emotions corresponding to the collateral conditions are outputted.

[0011] The states of the emotions generated by such a system arereflected on, for example, voices and images that are outputted.

SUMMARY OF THE INVENTION

[0012] However, the conventional emotion detecting method shows a lowprecision of detecting emotions, and cannot detect actual emotions of ahuman accurately even if it can detect emotions as to particularlylimited languages. Accordingly, the emotion detecting method is put topractical use only for limited use in, for example, a relatively simplegame machine.

[0013] It is an object of the present invention to provide an emotiondetecting method capable of accurately detecting emotions of a human whois a subject.

[0014] Furthermore, the conventional emotion generating method merelygenerates emotions directly based on information concerning situationsinputted. In actual humans, a variety of parameters including instinct,reason, individuality, and the like affect on one another complicatedly,resulting in variations of actions, speeches, expressions and the like.The conventional emotion generating method cannot precisely reflect theinstinct, reason, individuality and the like on the results.

[0015] Instinct and emotion can be regarded as affectivity. In addition,the instinct becomes basic biological affectivity and motivation of itsemotion generation. Furthermore, it is considered that humans do notdirectly output emotions, but they output sensibility controlled by thereason and the individuality.

[0016] It is another object of the present invention to provide asensibility generating method capable of outputting sensibility moreakin to that of a human.

[0017] According to a first aspect of the invention, an emotiondetecting method for detecting an emotion of a subject includes thefollowing steps: inputting a voice signal; detecting an intensity of avoice, a tempo expressing speed the voice emerges at, and intonationexpressing an intensity-change pattern in each word the voice makes,based on the inputted voice signal, respectively; obtaining amounts ofchange in the intensity of the voice detected, the tempo of the voice,and the intonation in the voice, respectively; and generating signalsexpressing states of emotion including at least anger, sadness, andpleasure, respectively, based on the obtained amounts of change.

[0018] In the first aspect of the invention, the emotion is detected byallowing the respective amounts of change in the intensity, tempo, andintonation of the voice inputted from the subject to correspond to thestates of emotion including anger, sadness, and pleasure, respectively.By using such a method, the emotion can be detected more precisely thanin the conventional art.

[0019] According to a second aspect of the invention, the emotiondetecting system for detecting an emotion of a subject includes: a voiceinputting unit for inputting a voice signal; an intensity detecting unitfor detecting an intensity of a voice based on the voice signal inputtedby the voice inputting unit; a tempo detecting unit for detecting speedthe voice emerges at as a tempo based on the voice signal inputted bythe voice inputting unit; an intonation detecting unit for detectingintonation expressing an intensity-change pattern in a word of the voicebased on the voice signal inputted by the voice inputting unit; achange-amount detecting unit for obtaining amounts of change in theintensity of the voice detected by the intensity detecting unit, thetempo of the voice detected by the tempo detecting unit, and theintonation in the voice detected by the intonation detecting unit,respectively; and an emotion detecting unit for outputting signalsexpressing states of emotion including at least anger, sadness, andpleasure, respectively, based on the amounts of change detected by thechange-amount detecting unit.

[0020] In the emotion detecting system of the second aspect of theinvention, the voice inputting unit, the intensity detecting unit, thetempo detecting unit, the intonation detecting unit, the change-amountdetecting unit, and the emotion detecting unit are provided, whereby theforegoing emotion detecting method can be executed.

[0021] According to a third aspect of the invention, the emotiondetecting system of the second aspect of the invention in which theintonation detecting unit includes: a bandpass filter unit forextracting specific frequency components from the voice signal which isinputted separately for each word; an area separating unit forseparating power spectrum of the signal which is extracted by thebandpass filter unit into a plurality of areas based on the intensity ofthe power spectrum; and an intonation calculating unit for calculating avalue of the intonation based on time intervals between respectivecenters of the plurality of areas separated by the area separating unit.

[0022] The bandpass filter unit extracts the specific frequencycomponents from the voice signal separated for each word and inputtedthereto. The area separating unit separates the detected power spectruminto the plurality of areas, based on the intensity thereof. Theintonation calculating unit calculates the value of the intonation basedon the time intervals between the respective centers of the plurality ofareas separated by the area separating unit.

[0023] In the third aspect of the invention, an energy distributionpattern in a word concerning the specific frequency components of thevoice is detected as a value of time expressing the intervals of theplurality of areas, and the length of the time is utilized as theintonation.

[0024] According to a fourth aspect of the invention, the emotiondetecting system of the second aspect of the invention further includes:an imaging unit for receiving image information concerning at least aface of the subject; an image recognition unit for detecting positionalinformation concerning each part of the face based on the imageinformation received by the imaging unit; an image reference informationretaining unit for retaining reference information concerning an amountof characteristic in each part of the face; and an image characteristicamount detecting unit for detecting an image characteristic amount basedon the positional information detected by the image recognition unit andthe reference information retained by the image reference informationretaining unit. The emotion detecting unit estimates a state of emotionaccording to a change in the image characteristic amount detected by theimage characteristic amount detecting unit.

[0025] In the fourth aspect of the invention, in addition to the voice,the state of emotion is estimated based on an expression of thesubject's face. Generally, since the states of emotion of humans arereflected on expressions of their faces, the states of emotion can begrasped by detecting the expressions of their faces. Accordingly, in thefourth aspect of the invention, the emotion detecting unit estimates thestate of emotion based on the change in the image characteristic amountdetected by the image characteristic amount detecting unit.

[0026] According to a fifth aspect of the invention, the emotiondetecting system of the second aspect of the invention further includes:an emotion information storing unit for sequentially receiving pieces ofinformation concerning the states of emotion detected by the emotiondetecting unit and for storing the pieces of information therein; and anoblivion processing unit for deleting information which has been storedfor a predetermined period of time since the information was initiallystored, among the pieces of information concerning states of emotionstored in the emotion information storing unit in the past, and forexcluding at least information showing a larger amount of change inemotion than a predetermined amount and information matching apredetermined change pattern, from the information to be deleted.

[0027] In the fifth aspect of the invention, it is possible to store theinformation concerning the detected states of emotion in the past in theemotion information storing unit. Furthermore, since the old informationwhich has been stored for a long period of time since its detection, isautomatically deleted from the emotion information storing unit, it ispossible to reduce storage capacitance required for the emotioninformation storing unit.

[0028] However, characteristic information such as the informationshowing a larger amount of change in emotion than the predeterminedamount and the information matching the predetermined change pattern areautomatically excluded from the information to be deleted. Therefore,the characteristic information is retained as it is in the emotioninformation storing unit even when it gets old. Accordingly, similarlyto a memory of a human, the characteristic information, which may beuseful in the future, can be read from the emotion information storingunit to be reproduced even when it gets old.

[0029] According to a sixth aspect of the invention, the emotiondetecting system of the fifth aspect of the invention further includes:a sentence recognition unit for executing grammar analysis by processinginformation concerning the voice uttered by the subject or charactersinputted by the subject, and for generating speech informationexpressing a meaning of a sentence; and a storage controlling unit forstoring the speech information generated by the sentence recognitionunit in the emotion information storing unit in synchronous with theinformation concerning the states of emotion.

[0030] The sentence recognition unit processes the informationconcerning the voice uttered by the subject or the characters inputtedby the subject with a keyboard or the like, and performs the grammaranalysis to generate the speech information expressing the meaning ofthe sentence.

[0031] The grammar analysis makes it possible to obtain the speechinformation expressing, for example, “5W3H”, that is, “Who”, “What”,“When” “Where”, “Why”, “How”, “How long, How far, How tall and so on”,and “How much”.

[0032] The storage controlling unit stores the speech informationgenerated by the sentence recognition unit in the emotion informationstoring unit in a state where the speech information is synchronous withthe information concerning the states of emotion.

[0033] In the sixth aspect of the invention, by referring to the emotioninformation storing unit, not only the information concerning theemotion at any time point in the past but also the speech informationexpressing situations at the time can be taken out.

[0034] The information retained in the emotion information storing unitcan be utilized in a variety of usages. For example, when an emotionestimating function of the emotion detecting system itself isinaccurate, a database which is used for estimating the emotion can becorrected based on the past result of detection retained in the emotioninformation storing unit.

[0035] According to a seventh aspect of the invention, the emotiondetecting system of the second aspect of the invention further includes:a voiceless time determining unit for determining a reference voicelesstime based on a state of emotion among the states of emotion detected;and a sentence segmentation detecting unit for detecting a segmentationof sentence of the voice by utilizing the reference voiceless timedetermined by the voiceless time determining unit.

[0036] When performing the recognition of the voice and the detection ofthe emotion, the segmentation for each sentence must be detected, andeach sentence must be extracted. In general, since a voiceless sectionexists in the segmentation between the sentences, a plurality ofsentences may be separated at timings when the voiceless sectionsappear.

[0037] However, the lengths of the voiceless sections are not constant.Particularly, the length of the voiceless section changes correspondingto the state of emotion of a speaker. Therefore, when a certainthreshold is allocated in order to determine the voiceless section, thepossibility of failure in detecting the segmentation of the sentencebecomes high.

[0038] In the seventh aspect of the invention, the reference voicelesstime is determined, for example, based on the state of emotion detectedjust before the determination, and the segmentation of sentence of thevoice is detected according to the reference voiceless time.Accordingly, it is possible to detect the segmentation of the sentencecorrectly even when the emotion of the speaker changes.

[0039] According to an eighth aspect of the invention, softwareincluding an emotion detecting program executable by a computer fordetecting an emotion of a subject in which the emotion detecting programincludes: a step of inputting a voice signal into the emotion detectingprogram; a step of detecting an intensity of a voice, a tempo expressingspeed the voice emerges at, and intonation expressing anintensity-change pattern in each word the voice makes, based on thevoice signal inputted; a step of obtaining amounts of change in each ofthe intensity of the voice, the tempo of the voice, and the intonationin the voice, which are detected; and a step of generating signalsexpressing states of emotion of at least anger, sadness, and pleasure,respectively, based on the obtained amounts of change.

[0040] It is possible to implement the emotion detecting method of thefirst aspect of the invention by executing, with a computer, the emotiondetecting program included in the software of the eighth aspect of theinvention.

[0041] According to a ninth aspect of the invention, a sensibilitygenerating method includes the steps of: retaining beforehand pieces ofindividuality information determining at least reason, a predeterminedcharacteristic, and will of a subject that generates sensibility;generating instinctive motivation information including at least a firstinstinct parameter expressing a degree of pleasure, a second instinctparameter expressing a degree of danger, and a third instinct parameterexpressing a degree of achievement and change, based on an inputtedsituation information which indicates a state of a partner's emotion oran environment the partner is in; generating emotion informationincluding a basic emotion parameter of at least pleasure, anger, andsadness, based on the instinctive motivation information generated; andcontrolling the emotion information generated based on the individualityinformation.

[0042] In the ninth aspect of the invention, the instinctive motivationinformation that motivates the generation of emotion is generated basedon the inputted situation information (the emotion, will, andcircumstance of the partner). Specifically, the instinctive motivationinformation is generated from the situation information, and the emotioninformation is generated based on the instinctive motivationinformation. Furthermore, the emotion information to be generated iscontrolled based on the individuality information. Therefore, theemotion controlled by the reason and will of the individual, that is,sensibility information, can be outputted.

[0043] In addition, since the emotion information is generated throughthe instinctive motivation information, the emotion to be generated canbe controlled more precisely and easily.

[0044] For example, an emotion generated when a human encounters adangerous situation in a state of already recognizing the dangeroussituation and an emotion generated when the human suddenly encountersthe dangerous situation in a state of not recognizing the danger at allare different. It is possible to reproduce such a difference in theemotions.

[0045] It is preferable to allow the instinct parameter to furtherinclude a degree of attention (degree of refusal), a degree of certainty(degree of puzzlement), a degree of follow-up (degree of assertion) andthe like in addition to the foregoing items. Furthermore, it ispreferable to allow the basic emotion parameter constituting the emotioninformation to further include surprise, fear, suffering, disgust,contempt, approach, escape, jealousy, envy, dependence, irritation,anxiety and the like in addition to the foregoing items.

[0046] According to a tenth aspect of the invention, a sensibilitygenerator includes: an instinct determining unit for inputting episodesituation information indicating states of a partner's emotion, anenvironment the partner is in, and the partner's will, and forgenerating instinctive motivation information including at least a firstinstinct parameter expressing a degree of pleasure, a second instinctparameter expressing a degree of danger, and a third instinct parameterexpressing a degree of achievement or change, based on the episodesituation information; an emotion generating unit for generating emotioninformation including basic emotion parameters of at least pleasure,anger, and sadness, based on the instinctive motivation informationoutputted from the instinct determining unit; an individualityinformation providing unit for providing individuality information whichdetermines at least reason and will with sensibility of a subject thatgenerates sensibility; and an emotion controlling unit for controllingemotion information outputted from the emotion generating unit, based onthe individuality information provided from the individualityinformation providing unit.

[0047] In the tenth aspect of the sensibility generator of theinvention, it is possible to execute the sensibility generating methodaccording to claim 9 by providing instinct determining unit, emotiongenerating unit, individuality information providing unit, and emotioncontrolling unit.

[0048] Accordingly, it is possible to output emotion controlled byreason and will of an individual, that is, information on sensibility.Furthermore, since emotion information is generated through instinctivemotivation information, emotion to be generated can be controlled moreprecisely and easily.

[0049] According to an eleventh aspect of the invention, the emotiongenerating unit of the tenth aspect of the invention includes: a liferhythm generating unit for generating information expressing anenvironment changing periodically or a life rhythm of a living body; anda voluntary emotion controlling unit for controlling voluntary emotionin the emotion generating unit according to the information on the liferhythm outputted by the life rhythm generating unit.

[0050] For example, natural environment conditions such as temperatureand humidity change periodically, though irregularly, concurrent withchanges of weather, season, time and the like. Furthermore, it isconsidered that respective humans have a rhythm of body, a rhythm ofemotion, a rhythm of intelligence and the like individually. The rhythmchanging periodically is considered to have various influences on theactual emotions of the humans.

[0051] In the eleventh aspect of the invention, the voluntary emotioncontrolling unit controls the voluntary emotion in the emotiongenerating unit according to the information on the life rhythmoutputted by the life rhythm generating unit. Accordingly, the emotionto be outputted can be changed in accordance with the environment or thelife rhythm of the living body.

[0052] According to a twelfth aspect of the invention, the sensibilitygenerator of the tenth aspect of the invention in which the emotiongenerating unit includes: an instinct-to-emotion information retainingunit for retaining pattern information which allows the basic emotionparameter and the instinctive motivation information to correspond toeach other; and a matching probability learning unit for outputtinginformation expressing a probability of matching/mismatching of theinstinctive motivation information with the pattern information of theinstinct-to-emotion information retaining unit, the instinctivemotivation information being outputted from the instinct determiningunit.

[0053] In the twelfth aspect of the invention, it is possible to obtainthe probability of matching of the instinctive motivation informationwith the pattern information from the matching probability learning unitto utilize it as a determination factor of the emotion.

[0054] For example, when a mental condition of a human changes from afirst state to a second state, the mental condition transits via a thirdstate on its way from the first state to the second state. Accordingly,there is a possibility that the mental condition temporarily be matchedwith certain pattern information in the third state. However, thepattern information matched with the mental condition in the third statedoes not show a value of high utility. By utilizing the probability ofthe matching obtained by the matching probability learning unit, thegeneration of emotion of the pattern information with a low probabilitycan be suppressed.

[0055] According to a thirteenth aspect of the invention, thesensibility generator of the tenth aspect of the invention in which theemotion generating unit includes an emotion feedback controlling unitfor inputting to the emotion generating unit at least its own emotioninformation finally generated, and for reflecting the finally generatedinformation on its own emotion information to be generated subsequently.

[0056] It is considered that inputting of various motivations causesemotion of a human to make chain changes. For example, a degree of angerwhich is emotion generated when a motivation is given to a person in anormal state so as to make him angry and a degree of anger which isemotion generated when a motivation is given to a person who has beenalready angry so as to make him further angry are different greatly fromeach other.

[0057] In the thirteenth aspect of the invention, the provision of theemotion feedback controlling unit allows the state of emotion generatedjust before the feedback to be brought back to an input and the state ofemotion to be reflected on an emotion to be generated subsequently.Accordingly, it is possible to generate an emotion more akin to that ofa human.

[0058] According to a fourteenth aspect of the invention, thesensibility generator of the tenth aspect of the invention has a featurein which the emotion controlling unit reflects information of a liferhythm, which is an individuality of a subject that generatessensibility, on the emotion information to be inputted.

[0059] In the fourteenth aspect of the invention, the information of thelife rhythm can be reflected on the sensibility. For example, adifference occurs in a result of determination made by reason and thelike, depending on whether a human is willing to do something. Such adifference in the sensibility can be reproduced by the reflection of thelife rhythm.

[0060] According to a fifteenth aspect of the invention, the sensibilitygenerator of the tenth aspect of the invention further includes: aknowledge database for storing situation information showing a pastsituation, a past episode, and a result of the past situation andepisode; a knowledge collating unit for retrieving and extracting pastsituation information analogous to newly inputted situation informationfrom the knowledge database, and for providing the past situationinformation to the emotion controlling unit; and a data updatecontrolling unit for updating contents of the knowledge database basedon the situation information showing a newly inputted situation and aresult of the new situation, and for automatically deleting, from theknowledge database, situation information of low priority in the orderof time in accordance with weight of the contents.

[0061] In the fifteenth aspect of the invention, the situationinformation showing the past situation and the result thereof is storedin the knowledge database. For example, information showing a situationof a certain episode and whether a final result of the episode hassucceeded is stored. Therefore, the situation information in the pastanalogous to that of the present situation can be acquired from theknowledge database to be utilized for controlling the emotion.

[0062] Incidentally, newly generated information must be addedsequentially to the knowledge database with elapse of time. However, astorage capacity of a system constituting the knowledge database islimited. Moreover, as an amount of the information stored is increased,a processing speed is lowered.

[0063] However, in the fifteenth aspect of the invention, the situationinformation of low priority is automatically deleted form the knowledgedatabase in the order of time, by the control of the data updatecontrolling unit. Therefore, a result similar to oblivion of a human canbe realized, and shortage of the storage capacity and lowering of theprocessing speed can be prevented.

[0064] According to a sixteenth aspect of the invention, the tenthaspect of the sensibility generator of the invention further includes: avoice inputting unit for inputting a voice signal; an intensitydetecting unit for detecting an intensity of a voice based on the voicesignal inputted by the voice inputting unit; a tempo detecting unit fordetecting speed the voice emerges at as a tempo based on the voicesignal inputted by the voice inputting unit; an intonation detectingunit for detecting intonation expressing an intensity-change pattern ina word of the voice, based on the voice signal inputted by the voiceinputting unit; a change-amount detecting unit for obtaining amounts ofchange in the intensity of the voice detected by the intensity detectingunit, the tempo of the voice detected by the tempo detecting unit, andthe intonation in the voice detected by the intonation detecting unit,respectively; and an emotion detecting unit for outputting signalsexpressing states of emotion of at least anger, sadness, and pleasure,respectively, based on the amounts of change detected by thechange-amount detecting unit.

[0065] In the sixteenth aspect of the invention, the partner's state ofemotion can be detected based on the amount of characteristic extractedfrom the voice. Accordingly, a self emotion in accordance with thepartner's emotion can be generated.

[0066] According to a seventeenth aspect of the invention, thesensibility generator of the sixteenth aspect of the invention furtherincludes: a voice recognition unit for recognizing the voice inputtedfrom the voice inputting unit, and for outputting character information;and a natural language processing unit for subjecting vocal informationrecognized by the voice recognition unit to natural language processing,and for generating meaning information expressing a meaning of theinputted voice.

[0067] In the seventeenth aspect of the invention, the meaninginformation concerning the word spoken by the partner is obtained, andthus a result obtained by understanding the meaning information can bereflected on the self sensibility.

[0068] According to an eighteenth aspect of the invention, softwareincluding a program and data executable by a computer utilized forsensibility generation control in which the program includes; a step ofgenerating instinctive motivation information including at least a firstinstinct parameter expressing a degree of pleasure, a second instinctparameter expressing a degree of danger, and a third instinct parameterexpressing a degree of achievement or change, based on an inputtedsituation information which indicates a state of a partner's emotion oran environment the partner is in; a step of generating emotioninformation including a basic emotion parameter of at least pleasure,anger, and sadness, based on the instinctive motivation informationgenerated; a step of providing individuality information determining atleast reason and will of a subject that generates sensibility; and astep of controlling the emotion information generated, based on theindividuality information.

[0069] The software of the eighteenth aspect of the invention isinputted to a predetermined computer to execute the program, and thusthe sensibility generating method of the ninth aspect of the inventioncan be implemented.

BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The nature, principle, and utility of the invention will becomemore apparent from the following detailed description when read inconjunction with the accompanying drawings in which like parts aredesignated by identical reference numbers, in which:

[0071]FIG. 1 is a block diagram illustrating a configuration of anemotion detecting system of an embodiment;

[0072]FIG. 2 is a block diagram illustrating a configuration of anintonation detecting unit;

[0073]FIG. 3 is a graph illustrating a relation between a change of anemotion state and an intensity, tempo, and intonation of a voice;

[0074]FIG. 4 is timing charts illustrating processes of a voice signalprocessing in the intonation detecting unit;

[0075]FIG. 5 is a flowchart illustrating an operation of an oblivionprocessing unit;

[0076]FIG. 6 is a schematic view illustrating a configuration example ofinformation stored in an emotion and sensibility memory DB;

[0077]FIG. 7 is a block diagram illustrating a configuration example ofa system using a sensibility generator;

[0078]FIG. 8 is a block diagram illustrating a configuration of aninstinct information generating unit;

[0079]FIG. 9 is a block diagram illustrating an emotion informationgenerating unit;

[0080]FIG. 10 is a schematic view illustrating an example of a reactionpattern model in an emotion reaction pattern DB; and

[0081]FIG. 11 is a block diagram illustrating a configuration of asensibility and thought recognition unit.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0082] (First Embodiment)

[0083] One embodiment relating to an emotion detecting method of thepresent invention will be described with reference to FIGS. 1 to 6.

[0084]FIG. 1 is a block diagram illustrating a configuration of anemotion detecting system of this embodiment. FIG. 2 is a block diagramillustrating a configuration of an intonation detecting unit. FIG. 3 isa graph illustrating a relation between a change of an emotion state andan intensity, tempo, and intonation of a voice. FIG. 4 is timing chartsillustrating processes of a voice signal processing in the intonationdetecting unit. FIG. 5 is a flowchart illustrating an operation of anoblivion processing unit. FIG. 6 is a schematic view illustrating aconfiguration example of information stored in an emotion andsensibility memory DB.

[0085] Referring to FIG. 1, the emotion detecting system comprises: amicrophone 11; an A/D converter 12; a signal processing unit 13; a voicerecognition unit 20; an intensity detecting unit 17; a tempo detectingunit 18; an intonation detecting unit 19; a temporary data storage unit21; an emotion change detecting unit 22; a voice emotion detecting unit23; an emotion pattern database (hereinafter referred to as DB) 24; akeyboard 25; a sentence recognition unit 26; a television camera 31; animage recognition unit 32; a face pattern DB 33; a face emotiondetecting unit 34; a character recognition unit 39; an emotion andsensibility memory DB 41; an oblivion processing unit 42; a synchronousprocessing unit 43; a humanity information DB 44; an individualityinformation DB 45; a specialty information DB 46; and an emotionrecognition unit 60.

[0086] Furthermore, in the voice recognition unit 20, provided are asignal processing unit 13; a phoneme detecting unit 14; a word detectingunit 15; and a sentence detecting unit 16. The voice recognition unit 20also includes a function of a voice recognizing (natural languageprocessing) device sold at stores.

[0087] In FIG. 1, the voice recognition unit 20, the intensity detectingunit 17, the tempo detecting unit 18, the intonation detecting unit 19,the temporary data storage unit 21, the emotion change detecting unit 22and the voice emotion detecting unit 23 are circuits for detecting anemotion from a voice.

[0088] The emotion detecting system comprises the microphone 11, thekeyboard 25 and the television camera 31 as inputting unit for readinginformation of a human who is a partner for which emotion is detected.Specifically, the emotion of the human who is the partner is detected byutilizing a voice inputted from the microphone 11, character informationinputted from the keyboard 25, and information including an expressionof a face and the like, which are inputted from the television camera31.

[0089] Note that the emotion can be also detected based solely on eitherof the voice inputted from the microphone 11, the character informationinputted from the keyboard 25 or the expression of the face inputtedfrom the television camera 31. However, from the viewpoint of improvinga detection precision of the emotion, it is more effective tocomprehensively judge the information obtained from the plurality ofinformation sources.

[0090] First, the processing relating to the voice will be described. Avoice signal inputted from the microphone 11 is sampled by the A/Dconverter 12, and then converted to a digital signal. The digital signalof the voice obtained at an output terminal of the A/D converter 12 isinputted to the voice recognition unit 20.

[0091] The signal processing unit 13 extracts frequency componentsnecessary for intensity detection of the voice. The intensity detectingunit 17 detects the intensity from the signal extracted by the signalprocessing unit 13. For example, a result obtained by averaging themagnitude of the amplitude of the voice signal can be used as theintensity.

[0092] An averaging cycle for detecting the intensity of the voice isset to about 10 seconds, for example. Note that, when segmentations forrespective sentences are detected in spite of an averaging cycle shorterthan 10 seconds, periods of time from the beginning of the sentence tothe detection of the segmentation are averaged. Specifically, theintensity of the voice for each sentence is detected.

[0093] The phoneme detecting unit 14 provided in the voice recognitionunit 20 detects segmentations for each phoneme of the voice inputtedthereto. For example, when the sentence expressed by “kyou wa ii tenkidesune” (in Japanese) is inputted in the form of a voice, thesegmentations for each phoneme like “kyo/u/wa/i/i/te/n/ki/de/su/ne” (inJapanese) are detected.

[0094] The word detecting unit 15 provided in the voice recognition unit20 detects segmentations for each word of the voice inputted thereto.For example, when the sentence expressed by “kyou wa ii tenki desune”(in Japanese) is inputted in the form of a voice, the segmentations foreach word like “kyou/wa/ii/tenki/desune” (in Japanese) are detected.

[0095] The sentence detecting unit 16 provided in the voice recognitionunit 20 detects segmentations for each sentence of the voice inputtedthereto. When a voiceless state of a specific length or more isdetected, it is considered that the segmentation for each sentenceappears. For a threshold value of the length of the voiceless state, thevalue of about 0.1 to 0.2 second is allocated. Moreover, this thresholdvalue is not constant, but this threshold value is changed automaticallyso that it reflects an emotion state detected immediately before.

[0096] The tempo detecting unit 18 receives the signal of thesegmentation for each phoneme outputted from the phoneme detecting unit14, and detects the number of phonemes that appeared at a unit time. Asto a detection cycle of the tempo, a time of about 10 seconds, forexample, is allocated. However, when the segmentation of the sentence isdetected, counting for the number of phonemes is stopped up until thetime point of the detection of the segmentation of the sentence even ifthe segmentation of the sentence is detected within 10 seconds, and avalue of the tempo is calculated. Specifically, the tempo is detectedfor each sentence.

[0097] The digital signal from the A/D converter 12 is divided for eachword in which the segmentations are detected by the word detecting unit15, and the voice signal is inputted to the intonation detecting unit19. From the voice signal inputted to the intonation detecting unit 19,the intonation detecting unit 19 detects the intonation expressing anintensity-change pattern of the voice in the word and in thesegmentation for each sentence in the sentence detecting unit 16. Thus,the intonation detecting unit 19 detects the characteristic intensitypattern in the segmentation.

[0098] As shown in FIG. 2, a bandpass filter 51, an absolute valueconversion unit 52, a comparison unit 53, an area center detecting unit54 and an area interval detecting unit 55 are provided in the intonationdetecting unit 19. Examples of the waveforms of the signals SG1, SG2,SG3 and SG4 of respective input or output terminals in the intonationdetecting unit 19 are illustrated in FIG. 4. Note that the ordinate ofeach signal in FIG. 4 indicates the amplitude or the intensity.Moreover, in the examples of FIG. 4, the length of one word taken outfrom the voice is about 1.2 seconds.

[0099] The bandpass filter 51 extracts only the frequency componentsnecessary for the detection of the intonation from the signal SG1inputted thereto. In this embodiment, only the frequency componentswithin the range of 800 to 1200 Hz appear at an output terminal of thebandpass filter 51 as the signal SG2. Referring to FIG. 4, it is foundthat the pattern of the intensity-change owing to the intonation in theword appears in the signal SG2.

[0100] In order to simplify calculation processing of the signal, theabsolute value conversion unit 52 is provided in the intonationdetecting unit 19. The absolute value conversion unit 52 converts theamplitude of the inputted signal to its absolute value. Accordingly, thesignal SG3 illustrated in FIG. 4 appears at an output terminal of theabsolute value conversion unit 52.

[0101] The comparison unit 53 compares the magnitude of the signal SG3with the threshold value, and outputs only components larger than thethreshold value as the signal SG4. Specifically, the comparison unit 53outputs only the components having large values in the power spectrum ofthe signal SG3. The threshold value applied to the comparison unit 53 isdetermined appropriately by a method called a decision analysis method.

[0102] Referring to FIG. 4, the two areas A1 and A2 corresponding tointonation pattern in the word of the voice appear in the signal SG4.The area center detecting unit 54 detects the times t1 and t2 at whichpositions corresponding to the respective centers of the two areas A1and A2 appear.

[0103] The area interval detecting unit 55 detects a time differenceconcerning the two times t1 and t2, which are detected by the areacenter detecting unit 54, as an area interval Ty. The value of this areainterval Ty corresponds to the intonation pattern in the word of thevoice. Actually, a result obtained by averaging the values of the areaintervals Ty is used as a value of the intonation.

[0104] In one word, three or more areas may appear in the signal SG4.When the three or more areas appear, the area intervals Ty arerespectively calculated for the two areas adjacent to each other, and aresult obtained by averaging the plurality of obtained area intervals Tyis used as the value of the intonation.

[0105] An emotion state of a human changes, for example, as illustratedin FIG. 3. Furthermore, in order to correctly grasp emotions includinganger, sadness, pleasure and the like, it is inevitable to detect achange of an amount of characteristic such as the intensity, the tempo,and the intonation.

[0106] In the emotion detecting system illustrated in FIG. 1, in orderto make it possible to refer to amounts of characteristic in the past,the value of the intensity outputted by the intensity detecting unit 17,the value of the tempo outputted by the tempo detecting unit 18 and thevalue of the intonation outputted by the intonation detecting unit 19are temporarily stored in the temporary data storage unit 21.

[0107] Furthermore, the emotion change detecting unit 22 receives thepresent value of the intensity outputted by the intensity detecting unit17, the present value of the tempo outputted by the tempo detecting unit18, and the present value of the intonation outputted by the intonationdetecting unit 19. The emotion change detecting unit 22 also receivesthe past (a little before the present time) values of the intensity, thetempo, and the intonation, which are stored in the temporary datastorage unit 21. Thus, the emotion change detecting unit 22 detects thechange of the state of emotion. Specifically, the emotion changedetecting unit 22 detects the changes in the intensity, tempo, andintonation of the voice, respectively.

[0108] The voice emotion detecting unit 23 receives the changes of theintensity, tempo, and intonation of the voice, which are outputted bythe emotion change detecting unit 22, and estimates the present state ofthe emotion. The voice emotion detecting unit 23 estimates three statesincluding anger, sadness and pleasure as the state of the emotion inthis embodiment.

[0109] In the emotion pattern DB 24, previously stored are informationallowing a state of the anger to relate to patterns of the changes ofthe intensity, tempo, and intonation of the voice, information allowinga state of the sadness to relate to patterns of the changes of theintensity, tempo, and intonation of the voice and information allowing astate of the pleasure to relate to patterns of the changes of theintensity, tempo, and intonation of the voice.

[0110] The voice emotion detecting unit 23 estimates the present stateof the emotion based on the patterns of the change of the intensity, thechange of the tempo and the change of the intonation, which areoutputted by the emotion change detecting unit 22, with reference to theinformation retained in the emotion pattern DB 24 as an estimation rule.

[0111] The information expressing the three types of states includingthe anger, the sadness and the pleasure, which have been estimated bythe voice emotion detecting unit 23, are inputted to the emotionrecognition unit 60 and the emotion and sensibility memory DB 41. Theemotion and sensibility memory DB 41 sequentially receives and storesthe present states of the emotion, which are inputted from the voiceemotion detecting unit 23.

[0112] Accordingly, the past state of the emotion can be reproduced byreading out the information stored in the emotion and sensibility memoryDB 41.

[0113] Meanwhile, the contents of the sentence inputted from themicrophone 11 as a voice (speech contents of the partner) is recognizedby the sentence recognition unit 26. The character informationcorresponding to the respective phonemes recognized by the voicerecognition unit 20 and the information expressing the segmentation ofthe word and the segmentation of the sentence are inputted to thesentence recognition unit 26. Moreover, the character informationinputted from the keyboard 25 is also inputted to the sentencerecognition unit 26.

[0114] The sentence recognition unit 26 recognizes an inputted characterstring for each word and analyzes the syntax thereof to grasp thecontents of the sentence as a natural language. Actually, the sentencerecognition unit 26 recognizes speech information expressing “5W3H”,that is, “Who”, “What”, “When” “Where”, “Why”, “How”, “How long, Howfar, How tall and so on” and “How much”. The speech informationrecognized by the sentence recognition unit 26 is inputted to theemotion recognition unit 60.

[0115] Next, processing for detecting the emotion based on a look on thepartner's face will be described. The television camera 31 photographsat least a facial part of the human who will be the subject of theemotion detecting system of FIG. 1. The image photographed by thetelevision camera 31, that is, an image including the look on the humanface is inputted to the image recognition unit 32.

[0116] Note that the information of the image photographed by thetelevision camera 31 is inputted to the character recognition unit 39.Specifically, the character recognition unit 39 recognizes therespective characters of a sentence from a photographed image when theimage of the sentence is photographed by the television camera 31. Thecharacter information recognized by the character recognition unit 39 isinputted to the sentence recognition unit 26.

[0117] The image recognition unit 32 recognizes characteristic elementsfrom the inputted image. Concretely, the image recognition unit 32recognizes respective parts of eyes, mouth, eyebrows, and cheekbones inthe face of the subject, and detects respective relative positions ofeyes, mouth, eyebrows and cheekbones in the face. Moreover, the imagerecognition unit 32 always traces the respective positions of eyes,mouth, eyebrows and cheekbones, in order to detect the respectivepositional changes thereof following the change of the facial look andto detect an expression such as shaking one's head.

[0118] Information concerning reference positions with regard to therespective positions of eyes, mouth, eyebrows, and cheekbones in theface (information equivalent to the facial look of the subject in anormal state thereof) is stored in advance in the face pattern DB 33.Note that it is also possible to change the contents of the face patternDB 33 arbitrarily. Moreover, rule information expressing correspondencerelationships between the changes of the facial look and six types ofemotions (pleasure, anger, sadness, fear, joy and surprise) is stored inadvance in the face pattern DB 33.

[0119] The face emotion detecting unit 34 detects the amounts ofcharacteristic of the look, that is, a difference thereof from that inthe normal state based on the information concerning the respectivepositions of eyes, mouth, eyebrows and cheekbones, which are recognizedby the image recognition unit 32, and the reference positions stored inthe face pattern DB 33.

[0120] Moreover, the face emotion detecting unit 34 estimates therespective states of the six types of emotions (pleasure, anger,sadness, fear, joy and surprise) based on the amounts of change andrates of the detected amounts of characteristic and on the ruleinformation retained in the face pattern DB 33. Information expressingthe estimated states of the six types of emotions is outputted from theface emotion detecting unit 34, and inputted to the emotion recognitionunit 60 and the emotion and sensibility memory DB 41.

[0121] The emotion recognition unit 60 comprehensively determines theinformation expressing the state of the emotion (anger, sadness orpleasure) inputted from the voice emotion detecting unit 23, the speechinformation inputted from the sentence recognition unit 26 and theinformation expressing the state of the emotion (pleasure, anger,sadness, fear, joy or surprise) inputted from the face emotion detectingunit 34. Then, the emotion recognition unit 60 estimates the final stateof the emotion. Regarding the speech information, the state of theemotion (pleasure, anger, sadness, fear, joy or surprise) includedtherein can be estimated by determining the contents (5W3H) of thesentence in the speech in accordance with a predetermined rule.

[0122] The information expressing the state of the emotion estimatedbased on the voice by the voice emotion detecting unit 23, theinformation concerning the speech contents recognized by the sentencerecognition unit 26 based on the voice or the characters inputted fromthe keyboard 25, and the information expressing the state of the emotionestimated from the facial look by the face emotion detecting unit 34,are respectively inputted to the emotion and sensibility memory DB 41and sequentially stored therein. Time and date when the respectivepieces of information stored in the emotion and sensibility memory DB 41are detected are added to the information.

[0123] Among the information inputted to the emotion and sensibilitymemory DB 41, the information concerning the emotion, which is inputtedfrom the voice emotion detecting unit 23, the information concerning thespeech contents, which is inputted from the sentence recognition unit26, and the information concerning the emotion, which is inputted fromthe face emotion detecting unit 34, must be grasped in association withone another.

[0124] Accordingly, the synchronous processing unit 43 associates theplural types of information stored in the emotion and sensibility memoryDB 41 with one another in accordance with the time (inputted time) anddate when such pieces of information are detected. For example, theinformation expressing the states of the emotions including anger,sadness and pleasure, which have been estimated by the voice emotiondetecting unit 23, and the information concerning the speech contents(5W3H), are associated with each other according to the points of timethereof as shown in FIG. 6.

[0125] Incidentally, the emotion and sensibility memory DB 41 includes asufficient storage capacity capable of storing a relatively large amountof information. However, since there are limitations on the storagecapacity, it is necessary to restrict the amount of information to bestored therein in order to use this system continuously for a longperiod of time.

[0126] In this connection, the oblivion processing unit 42 is provided.The oblivion processing unit 42 automatically deletes old informationfrom the emotion and sensibility memory DB 41. However, informationadapted to a specific condition is not deleted but stored even if itgets old.

[0127] An operation of the oblivion processing unit 42 will be describedwith reference to FIG. 5.

[0128] In Step S11 of FIG. 5, with regard to each of a large number ofdata stored in the emotion and sensibility memory DB 41, informationconcerning time and date when each of the data is stored (or detected)is referred to.

[0129] In Step S12, discrimination is made as to whether or not apredetermined certain period has elapsed since the relevant data wasstored, based on the current time and the time referred to in Step S11.In the case of processing old data that has been stored for a certainperiod since its storage time point, the processing proceeds to Step S13and after. Relatively new data that has not yet been stored for acertain period continues to be stored as it is.

[0130] In Step S13, when the data is information expressing the state ofthe emotion, the amount of change of the information (difference ofemotions before and after an event) is investigated. Since theprocessing proceeds from Step S13 to S17 when the amount of change ofthe emotion exceeds a predetermined threshold value, the data is storedas it is even when the data is old. When the amount of change of theemotion is equal to/less than the threshold value, the processingproceeds from Step S13 to S14.

[0131] In Step S14, the pattern of the emotion concerning the data isdetected, and discrimination is made as to whether or not the relevantpattern coincides with a predetermined specific pattern. Specifically,investigation is made as to whether or not plural combinations of thestates of the emotion and the speech contents coincide with a specificpattern representing a “strongly impressive” state. Since the processingproceeds from Step S14 to S17 when the detected pattern coincides withthe specific pattern, the data is stored as it is even if the data isold. When the patterns do not coincide with each other, the processingproceeds from Step S14 to S15.

[0132] In Step S15, when the data is the speech contents, discriminationis made as to whether or not the contents coincide with predeterminedspeech contents (significantly impressive speech). Even if both of thecontents do not coincide with each other completely, they can also beregarded to coincide when a similarity between the both is high. Sincethe processing proceeds from Step S15 to S17 when the speech contents ofthe relevant data coincide with the predetermined speech contents, thedata is stored as it is even if the data is old.

[0133] When both of the contents do not coincide with each other in StepS15, the relevant data is deleted in Step S16.

[0134] The above-described processing is executed for the entire data inthe emotion and sensibility memory DB 41. Moreover, the oblivionprocessing shown in FIG. 5 is executed periodically and repeatedly. Anexecution cycle of the oblivion processing can be arbitrarily changed asan individuality of an individual. Note that the processing is carriedout in Steps S14 and S15 by referring to a previously prepared patternDB (not shown). With regard to this pattern DB, contents thereof areautomatically updated by learning information inputted thereto.

[0135]FIG. 5 shows simplified processing. Actually, the entire of theamount of change of the emotion, the pattern of the emotion and thecontents of the speech are determined comprehensively. Specifically,when there exist the information in which the amount of change of theemotion is large, the information in which the pattern of the emotioncoincides with the specific pattern, and the information in which thespeech contents are the same or similar to the predetermined speechcontents, priorities thereof are determined comprehensively. Concretely,the information in which the speech contents are the same or similar tothe predetermined speech contents is given the highest priority, theinformation in which the pattern of the emotion coincides with thespecific pattern is given the second-highest priority, and theinformation in which the amount of change of the emotion is large isgiven the lowest priority. Accordingly, the information in which thespeech contents are the same or similar to the predetermined speechcontents is unlikely to be deleted in the oblivion processing, andremains as a memory even if it gets old.

[0136] With regard to the old data in the emotion and sensibility memoryDB 41, only the data in which the change of the emotion is large, thedata having the pattern regarded as “strongly impressive”, the datainputted repeatedly many times, and the data in which the speechcontents are significantly impressive, are added with priorities inaccordance with their strengths and contents and stored as they are bythe processing as described above in the oblivion processing unit 42.Consequently, the old data in the emotion and sensibility memory DB 41becomes incomplete data having only a part remaining therein. Such datahas contents similar to a past ambiguous memory in human memory.

[0137] The past state of the emotion and the past speech contents, whichhave been stored in the emotion and sensibility memory DB 41, are readout to be subjected to data analysis, thus, for example, making itpossible to determine whether or not the emotion detecting systemoperates normally and to update databases of the respective unitsutilized for estimating the emotion so as to improve the contentsthereof.

[0138] The data stored in the emotion and sensibility memory DB 41 arefurther allocated in accordance with their contents, and are stored inthe humanity information DB 44, the individuality information DB 45 orthe specialty information DB 46.

[0139] In the humanity information DB 44, there are stored informationdefining a character of the subject, such as sex, age, aggressiveness,cooperativeness and current emotion, and information concerning adecision pattern of an action. In the individuality information DB 45,information such as an address of individual, current situation, currentenvironment and speech contents (5W3H) is stored. In the specialtyinformation DB 46, information such as occupation, carrier, occupationalaptitude and occupational action decision pattern is stored.

[0140] What is outputted from the humanity information DB 44, theindividuality information DB 45 and the specialty information DB 46 ismoral pattern information of an individual. The sensibility of thepartner can be perceived based on the moral pattern information and thepast emotion of the partner.

[0141] When the function of the emotion detecting system shown in FIG. 1is realized by software of a computer, it is satisfactory when a programexecuted by the computer and necessary data may be previously recordedin a recoding medium such as, for example, a CD-ROM.

[0142] Note that the microphone 11 shown in FIG. 1 may be replaced by areceiver of a telephone, and that a mouse may be provided as unit forinputting information such as characters.

[0143] Moreover, the television camera 31 shown in FIG. 1 may bereplaced by any of various imaging unit such as an optical camera, adigital camera and a CCD camera.

[0144] The emotion of the subject can be detected more accurately thanthe conventional by using the emotion detecting method as describedabove.

[0145] (Second Embodiment)

[0146] Next, one embodiment relating to a sensibility generating methodof the present invention will be described with reference to FIGS. 7 to11.

[0147]FIG. 7 is a block diagram illustrating a configuration example ofa system using a sensibility generator. FIG. 8 is a block diagramillustrating a configuration of an instinct information generating unit.FIG. 9 is a block diagram illustrating an emotion information generatingunit. FIG. 10 is a schematic view illustrating an example of a reactionpattern model in an emotion reaction pattern DB. FIG. 11 is a blockdiagram illustrating a configuration of a sensibility and thoughtrecognition unit.

[0148] The system shown in FIG. 7 is configured on the assumption that anatural and sensible dialog between an arbitrary human and a computer(virtual human) is realized. In this example, an emotion detectingsystem 200 is provided in order to detect the emotion of the human whowill be a partner of the computer, and a sensibility generator 100 isprovided in order to reflect the individuality and sensibility of thecomputer itself on the dialog.

[0149] Moreover, an environmental information input device 300 isprovided in order to input a variety of environmental information to thesensibility generator 100. The environmental information input device300 outputs information concerning, for example, date, time, weather,location and image.

[0150] The sensibility generator 100 can also be utilized for a systemoperating autonomously. For example, when information concerning apreviously created scenario is inputted to the sensibility generator100, then a reaction in accordance with the scenario can be obtainedfrom the output of the sensibility generator 100. In this case, theemotion detecting system 200 is not required.

[0151] Although devices required for realizing the dialog are connectedto the output of the sensibility generator 100 in the example of FIG. 7,sensibility data outputted by the sensibility generator 100 can beutilized for various purposes.

[0152] For example, in the case of utilizing the sensibility generator100 in data communication, it is not necessary to output a voice sincecharacter information may satisfactorily be outputted. Moreover, thesensibility data outputted from the sensibility generator 100 can alsobe reflected on image, music, information retrieval and machine control.

[0153] Next, the configuration and operation of the sensibilitygenerator 100 will be described. Since the same one as the emotiondetecting system 200 of FIG. 1, which has been already described, isassumed for the emotion detecting system 200 in this embodiment,description thereof will be omitted.

[0154] Actually, the system shown in FIG. 7 can be composed of acomputer system and a software program executed therein, or can berealized as exclusive hardware. Moreover, the software program and datato be used can be stored in an arbitrary recording medium in advance,and can be read in the computer from the recording medium for execution.Note that the system itself of FIG. 7 is referred to as a computer inthe following description.

[0155] Roughly divided, two types of data, that is, data D1 and data D2are inputted to the input of the sensibility generator 100. The data D1is information expressing the emotion of the partner. The data D2 ischaracter information that has been subjected to natural languageprocessing, and includes information concerning the will, situation andenvironment of the partner. By the natural language processing, the dataD2 is inputted as information expressing the “5W3H”, that is, “Who”,“What”, “When” “Where”, “Why”, “How”, “How long, How far, How tall andso on” and “How much”.

[0156] Actually, it is possible to utilize a variety of information asbelow, as inputs to the sensibility generator 100.

[0157] (A) Change patterns of vocalism relating to temporal property,which includes stress, rhythm, tempo, pause, musical scale, musicalinterval, melody, harmony, frequency and the like; and degrees of basicemotions (anger, pleasure, sadness, disgust, surprise, fear and thelike)

[0158] (B) Information concerning vocalism relating to tonic property,which includes accent, depth, denseness, brightness, roughness, tonecolor (JIS-Z8109), formant, intonation, prominence for making a certainpart of a spoken language prominent to clarify a meaning, and the like

[0159] (C) Word, segment contents, stress distribution in sentence,suprasegmental characteristic information, characteristic informationgenerated by artificial intelligence, those of which relate to propertyof stress

[0160] (D) Text information subjected to conversation analysis, episodeinformation (including meaning information and information recognized byartificial intelligence) and the like.

[0161] Among such pieces of information, the information (A) and theinformation (B) are affected by intention and emotion of a speaker. Suchemotion can be detected by the emotion detecting system 200.

[0162] As shown in FIG. 7, the sensibility generator 100 includes aninstinct information generating unit 110, a metrical pattern DB 121, aninstinct language defining dictionary 122, an emotion informationgenerating unit 130, an emotion reaction pattern DB 141, a temporarystorage DB 142, a sensibility and thought recognition unit 150, aknowledge DB 161, a sensibility DB 162, an individual DB 163 and a moralhazard DB 164.

[0163] The function of the sensibility generator 100 can be basicallydivided into three functional elements of the instinct informationgenerating unit 110, the emotion information generating unit 130 and thesensibility and thought recognition unit 150. First, the instinctinformation generating unit 110 will be described.

[0164] As shown in FIG. 8, the instinct information generating unit 110includes a metrical-pattern matching recognition unit 111, an instinctparameter generating unit 112 and a dictionary retrieval unit 113.

[0165] A dictionary of metrical patterns inputted to the computer(virtual human) is stored in advance in the metrical pattern DB 121referred to by the metrical-pattern matching recognition unit 111. Themeter is a rhythmic element of a speech, and represents phonetic andphonological characteristics emerging for a syllable, a word, a phrase,a sentence and the entire speech (continuous voice longer than a word).Specifically, pattern information of the computer's own, which isequivalent to the inputted information (A) and (B), is stored asindividuality information in the metrical pattern DB 121.

[0166] The metrical-pattern matching recognition unit 111 comparespartner's emotion analysis data D1 inputted from the emotion detectingsystem 200 with the metrical pattern stored in the metrical pattern DB121, and recognizes synchronization and matching degrees of the both.Information expressing the presence of a strong tone and the emotionalchange emerges in the output of the metrical-pattern matchingrecognition unit 111.

[0167] Meanwhile, information concerning instinct stimulation isregistered in advance in the instinct language defining dictionary 122.Concretely, a variety of information expressing stress allocationpatterns and suprasegmental characteristics in a word or a sentence,which relate to the property of the stress, are stored as a dictionaryin association with the instinct stimulation.

[0168] The dictionary retrieval unit 113 compares data D2 inputted ascharacter information (will and situation of a partner) with thecontents of the instinct language defining dictionary 122, and generatesinstinctive reaction information from the contents of a conversation.

[0169] The instinct parameter generating unit 112 generates instinctivemotivation information D4 based on the information inputted from themetrical-pattern matching recognition unit 111, the information inputtedfrom the dictionary retrieval unit 113 and data D3. The data D3 isinformation feedbacked from the output of the sensibility generator 100,and has episode and desire reaction patterns proposed by the computer.

[0170] In this example, the instinctive motivation information D4includes six instinct parameters: a degree of certainty (or degree ofpuzzlement); a degree of pleasure (or degree of unpleasure); a degree ofdanger (or degree of safety); a degree of attention (or degree ofrefusal); a degree of achievement (or degree of change); and a degree offollow-up (or degree of assertion). The instinct parameter generatingunit 112 decides values of the respective instinct parameters in thefollowing manner.

[0171] Degree of pleasure (degree of unpleasure): when the computercomes close to proposed contents or a desired situation episode, thedegree of pleasure is increased, and otherwise, the degree is decreased.Moreover, when the computer comes close to a meter predetermined to bepleasant, the degree of pleasure is increased, and otherwise, decreased.

[0172] Degree of danger (degree of safety): when the computer comesclose to contents previously regarded as dangerous and a situationepisode assumed to be dangerous, the degree of danger is increased, andotherwise, decreased. Moreover, when the computer comes close to a meterpredetermined to be dangerous, the degree of danger is increased, andotherwise, decreased.

[0173] Degree of achievement (degree of change): when the computer comesclose to contents predetermined to be successful/achieved and asituation episode previously assumed to be successful/achieved, thedegree of achievement is increased, and otherwise, decreased. Moreover,when the computer comes close to a specific meter regarded as radicallymodulated, the degree of change is increased, and otherwise, decreased.

[0174] Degree of attention (degree of refusal): when the computer comesclose to contents previously regarded as refused/denied and a situationepisode previously assumed to be refused/denied, the degree of refusalis increased (the degree of attention is decreased), and otherwise,decreased (increased). Moreover, when the computer detects a strong orrepeated assertion or comes close to a strong meter, the degree ofattention is increased. When the computer comes close to a meterdetermined to be unpleasant, the degree of refusal is increased.

[0175] Degree of follow-up (degree of assertion): when the computercomes close to contents predetermined to be self-disparaging/self-denialand a situation episode previously assumed to beself-disparaging/self-denial, the degree of follow-up is increased(degree of assertion is decreased). When contents previously determinedto be good emerge, the degree of assertion is increased (degree offollow-up is decreased). Moreover, when a meter predetermined to beuncertain emerges, the degree of assertion is increased. Note that, whenthe computer comes close to a strong meter, a degree of repulsion or thedegree of self-denial may sometimes be increased.

[0176] Degree of certainty (degree of puzzlement): when the computercomes close to puzzled contents and an assumed situation episode, in thecase where a recognition rate of various stimuli (inputs) relating tothe instinct is low (for example, 70% or less), the degree of puzzlementoccurs in inverse proportion to the recognition rate. The recognitionrate is determined by a vocal tone and contents of a conversation.

[0177] In order to realize such control as described above, the contentsdesired by the computer and the meter of the situation episode arepreviously decided as individualities. As described above, the partner'semotion information stimulates the individual instinct of the computer,and thus the values of the respective instinct parameters are changed.

[0178] The instinctive motivation information D4 outputted from theinstinct information generating unit 110 is inputted to the emotioninformation generating unit 130. Next, the emotion informationgenerating unit 130 will be described.

[0179] As shown in FIG. 9, the emotion information generating unit 130includes a reaction pattern retrieval unit 134, a learning processingunit 135, a multivariate analysis unit 136, a voluntary emotion controlunit 137 and a basic emotion parameter generating unit 133.

[0180] The reaction pattern retrieval unit 134, the learning processingunit 135 and the emotion reaction pattern DB 141 compose a respondentsystem 131. The multivariate analysis unit 136 and the voluntary emotioncontrol unit 137 compose an operant system 132.

[0181] The respondent system 131 is provided in order to generate anemotion caused by stimulus induction. The operant system 132 is providedin order to generate a voluntary emotion (libido).

[0182] Information concerning a reaction pattern model representing acorrespondence relationship between the instinctive motivationinformation D4 and the basic emotion parameter is previously stored inthe emotion reaction pattern DB 141 for use in the respondent system131. This reaction pattern model can be shown, for example, as in FIG.10.

[0183] In the case of selectively reproducing personalities of aplurality of humans by one computer, reaction pattern models, eachcorresponding to each of the plurality of humans or each type ofindividualities thereof, are registered in advance in the emotionreaction pattern DB 141, and a reaction pattern model may be selected inaccordance with the individuality of the selected human.

[0184] In this example, the above-described six instinct parametersinputted as the instinctive motivation information D4 are assumed, whichare: the degree of certainty (or degree of puzzlement); the degree ofpleasure (or degree of unpleasure); the degree of danger (or degree ofsafety); the degree of attention (or degree of refusal); the degree ofachievement (or degree of change); and the degree of follow-up (ordegree of assertion).

[0185] As basic emotion parameters outputted from the emotioninformation generating unit 130, the following fifteen types ofparameters are assumed. The terms in the parentheses denote instinctparameters affected by the basic emotion parameters.

[0186] 1. Anger (unpleasure)

[0187] 2. Joy/cheerfulness (pleasure)

[0188] 3. Sadness (un-achievement/stagnation/unpleasure)

[0189] 4. Surprise (achievement/impact)

[0190] 5. Fear (danger/tension)

[0191] 6. Suffering (danger/tension/unpleasure)

[0192] 7. Disgust (rejection/refusal/unpleasure)

[0193] 8. Contempt (rejection/flaccidity)

[0194] 9. Approach (pleasure/safety)

[0195] 10. Escape/avoidance (danger/tension/unpleasure)

[0196] 11. Jealousy (unpleasure/anger/envy/attention)

[0197] 12. Positiveness (safety/pleasure/certainty)

[0198] 13. Dependence (achievement/follow-up)

[0199] 14. Irritation/conflict (assertion/stagnation/unpleasure/danger)

[0200] 15. Anxiety (danger/tension/puzzlement/unpleasure)

[0201] Reaction patterns representing relations with one or plural basicemotion parameters are stored for each of the fifteen types of basicemotion parameters in the emotion reaction pattern DB 141.

[0202] The reaction pattern retrieval unit 134 retrieves the reactionpatterns of the basic emotion parameters in the emotion reaction patternDB 141, investigates matching/mismatching thereof with the inputtedinstinctive motivation information D4, and outputs the information ofthe matched basic emotion parameters as data D6.

[0203] The learning processing unit 135 learns a probability regarding away of pattern matching based on the information D3 outputted from thesensibility and thought recognition unit 150 and the partner's nextreactive emotion outputted from the reaction pattern retrieval unit 134,and changes the contents of the emotion reaction pattern DB 141according to results of the learning.

[0204] Meanwhile, environment information (D2) including, for example,weather information, season information, time information and the likeis inputted to the input of the operant system 132. The multivariateanalysis unit 136 carries out multivariate analysis for a variety ofinputted environment information (D2), and consequently, outputs liferhythm information.

[0205] In the life rhythm information, there are regular (sine waveshaped) rhythms having constant cycles, such as a short-period rhythm(for example, one-hour cycle), a life rhythm (for example, 24hour-cycle), an emotion long-period rhythm (for example, 28 day-cycle),a body long-period rhythm (for example, 23 day-cycle) and anintelligence rhythm (for example, 33 day-cycle), and there are irregularrhythms such as temperature, humidity and weather.

[0206] The voluntary emotion control unit 137 outputs the voluntaryemotion (libido) among the life rhythm information outputted from themultivariate analysis unit 136 in accordance with a probability in apredetermined range.

[0207] The basic emotion parameter generating unit 133 outputs a resultobtained by comprehensively determining the information concerning thebasic emotion parameter and the matching rate, which are outputted fromthe respondent system 131 and the voluntary emotion outputted from theoperant system 132, as self emotion information D5. In this case, theresult is information composed of the fifteen types of basic emotionparameters.

[0208] Moreover, the outputted self emotion information DS istemporarily stored in the temporary storage DB 142, and feedbacked tothe input of the basic emotion parameter generating unit 133. The basicemotion parameter generating unit 133 receives the informationfeedbacked from the temporary storage DB 142 as a self emotionimmediately before, and reflects the same on an emotion determinationresult at the next time.

[0209] When the basic emotion parameter generating unit 133 carries outcomprehensive determination, it decides the priorities and degrees ofinfluences of the respective units in accordance with an individualitydetermined as individuality information 143.

[0210] For example, in the case of reproducing an impulse-type emotion,the degree of influence of the respondent system 131 is increased (80%or more), and the influence of the self emotion immediately before isalso increased. In the case of reproducing a thought-type emotion, thedegree of influence of the respondent system 131 is decreased (30% orless), and the influence of the self emotion immediately before is alsodecreased under an environment where the output of the operant system132 is stable.

[0211] The self emotion information D5 outputted from the emotioninformation generating unit 130 is inputted to the sensibility andthought recognition unit 150. As shown in FIG. 11, the emotioninformation generating unit 130 includes a weight-putting processingunit 151, a collation processing unit 152, a multivariate analysis unit153, a comprehensive intuitive decision-making unit 154 and an updatingprocessing unit 156.

[0212] The weight-putting processing unit 151 puts weight to theinputted self emotion information DS in accordance with individualityinformation 155. The weight-put self emotion information is outputtedfrom the weight-putting processing unit 151.

[0213] Meanwhile, character information (5W3H) including an episoderepresenting an environment and a situation a partner is in, and thepartner's will and a result thereof is inputted as the data D2 to theinput of the collation processing unit 152.

[0214] The past episode, the result thereof and the meaning informationexpressing their meanings are stored as knowledge in the form ofcharacter information (5W3H) in knowledge DB 161 referred to by thecollation processing unit 152. Moreover, the pieces of knowledge in theknowledge DB 161 include information of times when the respective dataare obtained, and are arrayed in accordance with the order of the times.

[0215] In this example, the pieces of knowledge in the knowledge DB 161can be classified into a long-term memory, a declarative memory and aprocedural memory. The declarative memory is a memory stored by words,and represents the episode information as events in a specifictemporal/spatial context and the meaning information as generalknowledge. The procedural memory represents memories regarding a methodand a technique.

[0216] The episode information includes time, place, contents, will(approval, opposition, favor and the like), person, quantity, weight,situation, state, partner's private information, affectivity, intention(object), attitude, personal relation and the like. The meaninginformation is equivalent to a language dictionary and a sensibilitydictionary. Conceived as the private information are temper, character,emotionality, social adaptability (sociability), desire, conflict,attitude, superiority, complex, interest, properness, morality, thoughtpattern, emotional particularity, persistence contents (and degreethereof), taboo word, taste, good/evil criterion, and the like.

[0217] In this example, the knowledge information is stored in theknowledge DB 161 in accordance with grammars as will be described below.However, the constituent contents of the database are changed accordingto objects.

[0218] Story Scene+Plot+Solution

[0219] Scene=Character+Place+Time

[0220] Theme=(Event)+Target

[0221] Plot=Episode

[0222] Episode=Lower target+Attempt+Result

[0223] Attempt=Event+Episode

[0224] Result=Event+State

[0225] Solution=Event+State

[0226] Lower target, Target=Desirable state

[0227] Character, Place, Time=State

[0228] Moreover, new information is sequentially added to the knowledgeDB 161 by the operation of the updating processing unit 156.Furthermore, unrequired information is automatically deleted from theknowledge by the oblivion processing performed repeatedly. Specifically,the data is sequentially deleted from the one getting older on the timebasis except the data having higher priorities. For example, priority isgiven to the knowledge utilized repeatedly and the data determined tohave a strong impression, and even if they get old, they are notdeleted. The degree of oblivion and the priorities of the respectivedata can be changed according to the individuality.

[0229] From the knowledge DB 161, the collation processing unit 152retrieves and extracts a past episode and a result thereof, which areclose to the inputted data D2, on the basis of the inputted data D2.Then, the collation processing unit 152 collates the inputted data withthe extracted knowledge.

[0230] A learning processing system 157 generates information concerningone's own concept of values for the inputted episode based on the resultthereof by learning. Specifically, the learning processing system 157puts degrees on satisfaction, pleasure and unpleasure from the result ofthe inputted episode.

[0231] The multivariate analysis unit 153 multivariately analyzes:weight-put emotion information inputted from the weight-puttingprocessing unit 151; the episode information and the result information,both of which are inputted from the collation processing unit 152; theinformation concerning the one's own concept of values, which isinputted from the learning processing system 157; and the informationconcerning the will and instinct of one's own, which is inputted fromthe individual DB 163. Then, the multivariate analysis unit 153 outputsthe result of the analysis to the comprehensive intuitivedecision-making unit 154.

[0232] The comprehensive intuitive decision-making unit 154 utilizes thecontents of the individual DB 163 and moral hazard DB 164 as adetermination dictionary, comprehensively determines the informationinputted from the multivariate analysis unit 153, and outputs what is tobe voluntarily executed and a result thereof as the data D3.

[0233] A variety of information as will be described below is stored asdictionary information in the individual DB 163.

[0234] 1. Individuality Information

[0235] (a) Determination criteria in accordance with degrees for eachtype of individuality: conceived as types are stereotype, other-orientedtype, inward-oriented type, tradition-oriented type, offense-orientedtype, cooperation-oriented type, stress-beating type, stress-releasingtype and the like. The degree of achievement motivation and the degreeof reactance can also be utilized as determination criteria.

[0236] (b) Determination criteria of cognitive styles: cognitive stylesby distinction between a “reflective type” and an “impulsive type” anddistinction between a “field-dependent type” and a “field-independenttype” are defined as determination criteria.

[0237] (c) Determination criteria by characters: in the case of theJapanese, the following that are classified by the personality testmethod and the TPI (Todai Personality Inventory) are utilized asdetermination criteria. The classified ones are: temper, character,emotionality, social adaptability (sociability), desire, conflict,attitude, complex, interest, properness, morality, thought pattern,emotional particularity, persistence contents (and degree thereof),taboo word, taste, good/evil criterion, shame criterion, sin criterion,criterion of pleasure and unpleasure, and the like.

[0238] (d) Determination criteria of negativity/bias: a bias is given tonegative information in order to grasp the same negative informationlargely, which is then utilized for forming a character.

[0239] (e) Determination criteria of adhesion/persistence time: a degreeof persistence for partner's cognitive information, episode and emotioninformation and a reaction correspondence time therefor are decided.

[0240] 2. Id/Unconscious Reaction Reference Information:

[0241] (a) Word dictionary and clause dictionary, each having contentsthat stimulate instincts.

[0242] (b) References of a variety of instinct reaction times for adegree of perseverance, a degree of adhesion and a degree ofstraightforwardness for each individuality. (c) Self instinct patterncorresponding to a partner's emotion decided as individuality.

[0243] 3. Reference information of homeostasis (inhibition):determination criteria for attempting to maintain the entire instinctoutputs to be stable in harmony.

[0244] 4. Self-conscious reaction reference time: information ofdetermination criteria representing one's own will by individuality.

[0245] Moreover, in the determination dictionary, included are:information utilized for recognition determination and identificationdetermination such as true/false, correct/incorrect andadequate/inadequate; information utilized for instinct determinationsuch as pleasure/unpleasure; information utilized for individualcognitive determination for subjects, such as complicatedness, weightand the like; information utilized for relative cognitive determinationbetween subjects, such as equality, size, difference and similarity;information utilized for meta-memory determination such as a degree ofcertainty for memory and accurateness of knowledge; information utilizedfor abstract determination such as truth, virtue, love and the like;information utilized for inductive determination; and the like.

[0246] Dictionary information concerning occupational moral, individualmoral, basic moral and the like is stored in the moral hazard DB 164.

[0247] For example, as the occupational moral, “As an architect, Irequire a complete calculation”, “I put the highest priority to my job”,“I have a pride that I am a professional” and the like are registered.Moreover, as the individual moral, “I value women (I do not boss aroundmen)”, “I am proud of my hometown”, “I am proud that I am Japanese” andthe like are registered. As the basic moral, “Killing a man is bad”, “Itake good care of my parents”, “I am a man (woman)” and the like areregistered.

[0248] The comprehensive intuitive decision-making unit 154 analyzes theinformation concerning the self emotion, which is generated by theemotion information generating unit 130, by the weight-puttingprocessing unit 151, the collation processing unit 152 and themultivariate analysis unit 153. Then, the comprehensive intuitivedecision-making unit 154 inhibits the analyzed information concerningthe self emotion based on the determination dictionary in the individualDB 163, which represents the individuality and will of this computer,and on the determination dictionary in the moral hazard DB 164.Subsequently, the comprehensive intuitive decision-making unit 154decides to what, what kind of and how much self emotional reaction(sensibility) is to be outputted. In the case of this decision, anenvironment and a situation a partner is in, and the partner's will atthat time are reflected.

[0249] The sensibility and thought recognition unit 150 includesfunctions as will be described below.

[0250] 1. In the case of detecting a strong impression or vocabulary ora radical emotion change, a determination cycle is changed according tothe individuality. For example, when strong contents are suddenlyasserted in a loud voice, the determination cycle is shortened.

[0251] 2. In response to one's own biorhythm depending on theindividuality, sensibility determination is carried out differentlydepending on whether or not one is willing to do something.

[0252] 3. In accordance with one's own pleasure/unpleasure and an amountof emotion, different sensibility determination is carried out.

[0253] 4. Reasonable value judgment is carried out for informationexpressing the present situation according to the knowledge on theknowledge DB 161, the influence of the judgment result of the emotion isreflected, and thus a final will is decided.

[0254] 5. When value judgment is carried out, the judgment is made fromthe respective viewpoints of a social value, an occupational value, adaily-life value, an individual value and the like. Moreover, each ofthe social value, the occupational value, the daily-life value and theindividual value is distinguished in more detail, and the judgment ismade. For example, with regard to the social value, values arecalculated from the respective viewpoints of religion, aesthetics,society, politics, economy and ethics.

[0255] 6. Value judgment is carried out for respective factors such assatisfaction/dissatisfaction, loss and gain/interests, safety/danger andthe like as judgment materials for the will decision. When the valuejudgment regarding the safety is carried out, for example, judgment ismade in a manner as described below.

[0256] (a) When a third person is to apply “unpleasure” to a self,values regarding a hostile emotion and a defense reaction are generated.

[0257] (b) When the self is to apply the “unpleasure” to the thirdperson, values regarding the hostile emotion and an offense reaction aregenerated.

[0258] (c) When the self is to take the third person's side when someother one is to apply the “unpleasure” to the third person, valuesregarding a favor emotion and a cooperative offense reaction aregenerated.

[0259] 7. The generated value information is stored in the sensibilityDB 162, and utilized as judgment materials thereafter.

[0260] Note that, since the sensibility and thought recognition unit 150includes a variety of learning functions similar to those of a human,the contents of the individual DB 163 and the sensibility DB 162 aresequentially updated by building up an experience.

[0261] Since the sensibility and thought recognition unit 150 outputsresults by comprehensive determination based on numerical values such asa variety of values, it does not carry out logical inference ordetermination as an artificial intelligence does. Specifically, the dataD3 outputted from the sensibility and thought recognition unit 150 issensibility information obtained by intuitive determination of thecomputer itself.

[0262] As described above, in the sensibility generating method of thepresent invention, the instinctive motivation information serving asmotivation for generating the emotion is generated based on the inputtedsituation information (partner's emotion, peripheral situation and thelike), and the emotion information is generated based on the instinctivemotivation information. Furthermore, the generated emotion informationis controlled in accordance with the individuality information.

[0263] Therefore, an emotion controlled by the reason and will of theindividuality, that is, the sensibility information can be outputted.Moreover, since the emotion information is generated through theinstinctive motivation information, the generated emotion can becontrolled more precisely and easily.

[0264] The emotion detecting method according to the present inventioncan be utilized for emotion detection in a medical field and can also beutilized in a variety of systems as a part of an artificial intelligenceor an artificial sensibility. Moreover, for sensibility control for avirtual human or robot, the sensibility generating method of the presentinvention can be utilized in a variety of systems for a variety ofpurposes. Furthermore, a variety of systems, each including a dialogfunction between a computer and a human, can be configured by combiningthe emotion detecting method and sensibility generating method of thepresent invention.

[0265] The invention is not limited to the above embodiments and variousmodifications may be made without departing from the spirit and scope ofthe invention. Any improvement may be made in part or all of thecomponents.

What is claimed is:
 1. An emotion detecting method for detecting anemotion of a subject, comprising the steps of: inputting a voice signal;detecting an intensity of a voice, a tempo expressing speed the voiceemerges at, and intonation expressing an intensity-change pattern ineach word the voice makes, based on the inputted voice signal; obtainingamounts of change in the intensity of the detected voice, tempo of thevoice, and intonation in the voice, respectively; and generating signalsexpressing states of emotion of at least anger, sadness, and pleasure,respectively, based on the obtained amounts of change.
 2. An emotiondetecting system for detecting an emotion of a subject, comprising: avoice inputting unit for inputting a voice signal; an intensitydetecting unit for detecting an intensity of a voice based on the voicesignal inputted by said voice inputting unit; a tempo detecting unit fordetecting speed the voice emerges at as a tempo based on the voicesignal inputted by said voice inputting unit; an intonation detectingunit for detecting intonation expressing an intensity-change pattern ina word of the voice based on the voice signal inputted by said voiceinputting unit; a change-amount detecting unit for obtaining amounts ofchange in the intensity of the voice detected by said intensitydetecting unit, the tempo of the voice detected by said tempo detectingunit, and the intonation in the voice detected by said intonationdetecting unit, respectively; and an emotion detecting unit foroutputting signals expressing states of emotion of at least anger,sadness, and pleasure, respectively, based on the amounts of changedetected by said change-amount detecting unit.
 3. The emotion detectingsystem according to claim 2, wherein said intonation detecting unitincludes: a bandpass filter unit for extracting specific frequencycomponents from the voice signal inputted separately for each word; anarea separating unit for separating a power spectrum of the signalextracted by said bandpass filter unit into a plurality of areas basedon the intensity of the power spectrum; and an intonation calculatingunit for calculating a value of the intonation based on time intervalsbetween respective centers of the plurality of areas separated by saidarea separating unit.
 4. The emotion detecting system according to claim2, further comprising: an imaging unit for receiving image informationconcerning at least a face of the subject; an image recognition unit fordetecting positional information concerning each part of the face fromthe image information received by said imaging unit; an image referenceinformation retaining unit for retaining reference informationconcerning an amount of characteristic in each part of the face; and animage characteristic amount detecting unit for detecting an imagecharacteristic amount based on the positional information detected bysaid image recognition unit and the reference information retained bysaid image reference information retaining unit, and wherein saidemotion detecting unit estimates a state of emotion according to achange in the image characteristic amount detected by said imagecharacteristic amount detecting unit.
 5. The emotion detecting systemaccording to claim 2, further comprising: an emotion information storingunit for sequentially receiving pieces of information concerning thestates of emotion detected by said emotion detecting unit and forstoring the pieces of information therein; and an oblivion processingunit for deleting information which has been stored for a predeterminedperiod of time since the information was initially stored, among thepieces of information concerning states of emotion stored in saidemotion information storing unit in the past, and for excluding at leastinformation showing a larger amount of change in emotion than apredetermined amount and information matching a predetermined changepattern, from said information to be deleted.
 6. The emotion detectingsystem according to claim 5, further comprising: a sentence recognitionunit for executing grammar analysis by processing information concerningany of the voice uttered by the subject and characters inputted by thesubject, and for generating speech information expressing a meaning of asentence; and a storage controlling unit for storing the speechinformation generated by said sentence recognition unit in the emotioninformation storing unit, in synchronous with the information concerningsaid states of emotion.
 7. The emotion detecting system according toclaim 2, further comprising: a voiceless time determining unit fordetermining a reference voiceless time based on a state of emotion amongthe detected states of emotion; and a sentence segmentation detectingunit for detecting a segmentation of sentence of the voice by utilizingthe reference voiceless time determined by said voiceless timedetermining unit.
 8. Software including an emotion detecting programexecutable by a computer for detecting an emotion of a subject, whereinsaid emotion detecting program includes: a step of inputting a voicesignal; a step of detecting an intensity of a voice, a tempo expressingspeed the voice emerges at, and intonation expressing anintensity-change pattern in each word the voice makes, based on theinputted voice signal; a step of obtaining amounts of change in theintensity of the detected voice, tempo of the voice, and intonation inthe voice, respectively; and a step of generating signals expressingstates of emotion of at least anger, sadness, and pleasure,respectively, based on the obtained amounts of change.
 9. A sensibilitygenerating method, comprising the steps of: retaining beforehand piecesof individuality information determining at least reason, apredetermined characteristic, and will of a subject that generatessensibility; generating instinctive motivation information including atleast a first instinct parameter expressing a degree of pleasure, asecond instinct parameter expressing a degree of danger, and a thirdinstinct parameter expressing a degree of achievement and change, basedon an inputted situation information which indicates a state of any of apartner's emotion and an environment the partner is in; generatingemotion information including a basic emotion parameter of at leastpleasure, anger, and sadness, based on said instinctive motivationinformation generated; and controlling said emotion informationgenerated based on said individuality information.
 10. A sensibilitygenerator, comprising: an instinct determining unit for inputtingepisode situation information indicating a partner's emotion, anenvironment the partner is in, and the partner's will, and forgenerating instinctive motivation information which indicates at least afirst instinct parameter expressing a degree of pleasure, a secondinstinct parameter expressing a degree of danger, and a third instinctparameter expressing one of a degree of achievement and a degree ofchange, based on said episode situation information; an emotiongenerating unit for generating emotion information including basicemotion parameters of at least pleasure, anger, and sadness, based onthe instinctive motivation information outputted from said instinctdetermining unit; an individuality information providing unit forproviding individuality information determining at least reason and willwith sensibility of a subject that generates sensibility; and an emotioncontrolling unit for controlling emotion information outputted from saidemotion generating unit, based on the individuality information providedfrom said individuality information providing unit.
 11. The sensibilitygenerator according to claim 10, wherein said emotion generating unitincludes: a life rhythm generating unit for generating informationexpressing any of an environment changing periodically and a life rhythmof a living body; and a voluntary emotion controlling unit forcontrolling voluntary emotion in said emotion generating unit, accordingto the information on the life rhythm outputted by said life rhythmgenerating unit.
 12. The sensibility generator according to claim 10,wherein said emotion generating unit includes: an instinct-to-emotioninformation retaining unit for retaining pattern information whichallows said basic emotion parameter and said instinctive motivationinformation to correspond to each other; and a matching probabilitylearning unit for outputting information expressing a probability ofmatching/mismatching of the instinctive motivation information with thepattern information of said instinct-to-emotion information retainingunit, the instinctive motivation information being outputted from saidinstinct determining unit.
 13. The sensibility generator according toclaim 10, wherein said emotion generating unit includes: an emotionfeedback controlling unit for inputting to the emotion generating unitat least its own emotion information finally generated, and forreflecting the finally generated information on its own emotioninformation to be generated subsequently.
 14. The sensibility generatoraccording to claim 10, wherein said emotion controlling unit reflectsinformation of a life rhythm, which is an individuality of a subjectthat generates sensibility, on the emotion information to be inputted.15. The sensibility generator according to claim 10, further comprising:a knowledge database for storing situation information showing a pastsituation, a past episode, and a result of the past situation andepisode; a knowledge collating unit for retrieving and extracting pastsituation information analogous to newly inputted situation informationfrom said knowledge database, and for providing the past situationinformation to said emotion controlling unit; and a data updatecontrolling unit for updating contents of said knowledge database basedon the situation information showing a newly inputted situation and aresult of the new situation, and for automatically deleting, from saidknowledge database, situation information of low priority in the orderof time in accordance with weight of the contents.
 16. The sensibilitygenerator according to claim 10, further comprising: a voice inputtingunit for inputting a voice signal; an intensity detecting unit fordetecting an intensity of a voice based on the voice signal inputted bysaid voice inputting unit; a tempo detecting unit for detecting speedthe voice emerges at as a tempo based on the voice signal inputted bysaid voice inputting unit; an intonation detecting unit for detectingintonation expressing an intensity-change pattern in a word of the voicebased on the voice signal inputted by said voice inputting unit; achange-amount detecting unit for obtaining amounts of change in theintensity of the voice detected by said intensity detecting unit, thetempo of the voice detected by said tempo detecting unit, and theintonation in the voice detected by said intonation detecting unit,respectively; and an emotion detecting unit for outputting signalsexpressing states of emotion of at least anger, sadness, and pleasure,respectively, based on the amounts of change detected by saidchange-amount detecting unit.
 17. The sensibility generator according toclaim 16, further comprising: a voice recognition unit for recognizingthe voice inputted from said voice inputting unit, and for outputtingcharacter information; and a natural language processing unit forsubjecting vocal information recognized by said voice recognition unitto natural language processing, and for generating meaning informationexpressing a meaning of the inputted voice.
 18. Software including aprogram and data executable by a computer utilized for sensibilitygeneration control, wherein said program includes: a step of generatinginstinctive motivation information including at least a first instinctparameter expressing a degree of pleasure, a second instinct parameterexpressing a degree of danger, and a third instinct parameter expressingany of a degree of achievement and change, based on an inputtedsituation information which indicates a state of any of a partner'semotion and an environment the partner is in; a step of generatingemotion information including a basic emotion parameter of at leastpleasure, anger, and sadness, based on said instinctive motivationinformation generated; a step of providing individuality informationdetermining at least reason and will of a subject that generatessensibility; and a step of controlling said emotion informationgenerated, based on said individuality information.